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Abstract 

This paper stands at the intersection of two distinct lines of research. One line is "holographic 
algorithms," a powerful approach introduced by Valiant for solving various counting problems in computer 
science; the other is "normal factor graphs," an elegant framework proposed by Forney for representing 
codes defined on graphs. We introduce the notion of holographic transformations for normal factor 
graphs, and establish a very general theorem, called the generalized Holant theorem, which relates a 
normal factor graph to its holographic transformation. We show that the generalized Holant theorem 
on the one hand underlies the principle of holographic algorithms, and on the other hand reduces to a 
general duality theorem for normal factor graphs, a special case of which was first proved by Forney. In 
the course of our development, we formalize a new semantics for normal factor graphs, which highlights 
various linear algebraic properties that potentially enable the use of normal factor graphs as a linear 
algebraic tool. 

1 Introduction 

The formulation of codes on graphs and the invention of iterative decoding algorithms for graphical codes 
have undoubtedly revolutionized coding theory. In this research area, the introduction of normal graphs [1] 
and their duality properties are arguably one of the most elegant and profound results. 

In his celebrated paper pQ, Forney introduced the notion of normal realizations, represented by normal 
graphs, as generalized state realizations of codes. In a normal graph, each vertex represents a local (group) 
code constraint, and each edge represents a variable that is involved in either one or two of the constraints. A 
variable involved in two constraints, called a state variable, is represented by a regular edge, namely, an edge 
connecting two vertices; the two vertices correspond to the two constraints involving the variable. A variable 
involved in only one constraint, called a symbol variable, is represented by a "dangling edge," 1 namely, an 
edge incident on only one vertex; the vertex corresponds to the single constraint involving the variable. The 
global behavior represented by the normal graph is the set of all symbol-state configurations satisfying all 
local constraints, and the realized code is the set of all symbol configurations that participate in at least 
one symbol-state configuration in the global behavior. Forney showed in [1] that codes realized by Tanner 
graphs [5] or Wiberg-type graphs [3J 2] , in which a variable may in general be involved in an unrestricted 
number of constraints, can be converted to normal graphs by properly replicating some variables. 

1 In Forney [T] , such edges are called "half-edges" . 



Normal realizations of codes have a fundamental duality property. By introducing a simple local "dual- 
ization" procedure that converts each local code in a normal realization to its dual code and inserts additional 
"sign inverters," Forney proved a normal graph duality theorem pQ, which shows that the dualized normal 
graph realizes the dual code. 

The notion of a normal graph may be extended to the notion of a normal factor graph, or Forney- 
style factor graph [5] [5] , which uses an identical graphical representation, but in which the graph vertices 
no longer represent constraints. Instead, in a normal factor graph, each vertex represents a local function 
involving precisely the variables represented by the edges incident to the vertex. Treated as a factor graph [7] 
with particular variable-degree restrictions, and interpreted using the standard semantics of factor graphs, 
a normal factor graph represents a multivariate function that factors as the product of all of the local 
functions that are represented by the graph vertices. When each local function in the normal factor graph 
is the indicator function of a local code constraint, the represented function is the indicator function of the 
global behavior. This makes normal factor graph representations of codes equivalent to normal graphs, and 
allows the translation of the normal graph duality theorem to an equivalent theorem for normal factor graphs 
that represent codes. 

Among the first to recognize the profound value of normal graphs, Koetter [5] applied normal graphs (or 
normal factor graphs for codes) and the duality theorem in a study of trellis formations [3] , which gives a 
necessary and sufficient condition of mergcability and a polynomial-time algorithm for deciding whether a 
trellis formation contains mcrgcable vertices. In addition, Koetter et al. [10] extended the applications of 
normal graphs from channel codes to network codes, and used the fundamental duality theorem to establish 
a reversibility theorem in network coding. 

In a seemingly distant research area of complexity theory, Valiant proved the tractability of families of 
combinatorial problems for which no polynomial-time solvers were known previously in a ground-breaking 
work [11] . In this paper, Valiant develops a very powerful family of algorithms, which he calls holographic 
algorithms, to solve such problems. Holographic algorithms are based on the concept of "holographic reduc- 
tion," and are governed by a fundamental theorem that Valiant calles the Holant theorem. 

Although in his original work [IT] Valiant dealt only with transforming a product of functions to a 
specific form, the Holant theorem establishes a principle for transforming an arbitrary product of functions 
to another product of functions such that the sum over the configuration space is unchanged. Consequently, 
when computing the sum of a product of functions, the Holant theorem provides a family of transformations 
that convert the product to a different one, for which the sum may be efficiently computable. 

Since many problems in coding and information theory require the computation of sums of products 
(such as in decoding error correction codes and in computing certain capacities), holographic algorithms 
become a potentially powerful tool for the information theory community. Indeed, in |12| . Schwartz and 
Bruck showed that certain constrained-coding capacity problems may be solved in polynomial time using 
holographic algorithms. 

This paper stands at the intersection of the above-mentioned two lines of research, bridging the two 
areas by unifying Valiant's holographic reduction and Forney's normal-graph dualization with the notion of 
"holographic transformations," a term that we coin in this paper. The focus of this paper will be on general 
normal factor graphs, in which the vertices may represent arbitrary functions rather than just indicator 
functions. 
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We first introduce a new semantics for normal factor graphs, which we call the "exterior-function se- 
mantics." In the exterior-function semantics, instead of letting the graph represent a product of the local 
functions, we let it represent a sum of products of local functions, which we call the "exterior function." In 
this setting, a normal factor graph may be viewed as an expression, or realization, of its exterior function 
in terms of a "sum of products." In fact, the notion of "sum of products" is the key connection between 
Forney's normal graph duality theorem and Valiant's Holant theorem: in the case of Valiant this "sum 
of products" is the number of configurations that needs to be computed, and in the case of Forney [T] (when 
using normal factor graphs rather than normal graphs to represent codes), this "sum of products" is a code 
indicator function (possibly up to a scale factor). In this new framework, a holographic transformation 
is defined as a transformation of a normal factor graph that changes all local functions subject to certain 
conditions, and converts the normal factor graph to a structurally identical one. 

The main result of this paper is what we call the generalized Holant theorem, relating a normal factor 
graph and its holographic transformations in terms of their realized functions. On the one hand, we show 
that the Holant theorem in [TT] is a special case of the generalized Holant theorem. On the other hand, we 
prove a general duality theorem for normal factor graphs as a corollary of the generalized Holant theorem. 
This duality theorem reduces to Forney's original normal graph duality theorem for normal factor graphs 
that represent codes. 

Another result of our development is a new understanding of normal factor graphs from a linear algebraic 
perspective. More specifically, the exterior-function semantics associates a normal factor graph with a "sum- 
of-products" form, and a sum-of-products form may be regarded as a linear algebraic expression such as a 
vector dot product, matrix product or tensor product. In contrast to conventional notations for algebraic 
expressions, in which re-ordering terms results in illegitimate or different mathematical expressions, sum-of- 
products forms and their normal factor graph representations eliminate the need to properly order terms 
(i.e., the local functions). This allows a transparent development of our results. To us, normal factor graphs 
with the exterior-function semantics appear to be a very natural and intuitive language for linear algebra, 
and may potentially be useful in a wide variety of applications. 

The holographic transformations introduced in this paper equip normal factor graphs with a rich family of 
linear transformations, potentially enabling normal factor graphs to serve as a more general analytic frame- 
work and computational tool. The power of these transformations, in addition to providing a fundamental 
duality theorem in coding theory, has also been hinted at by the great power of holographic algorithms (see, 

e.g., puna nam mama). 

This paper was initially motivated by our recognition of the connection between Forney's normal graph 
duality theorem and Valiant's Holant theorem. Interactions with the editors of this paper (G. D. Forney, Jr. 
and P. Vontobel) and communications with Forney on his concurrent development of "partition functions 
of normal factor graphs" |17j have also provided important inspirations for our development. Indeed, much 
of our development shares similarities with Forney's approach in |17j . which also proves the general normal 
factor graph duality theorem. 

During the review process, we learned from the reviewers and the editors of this paper that the techniques 
used to establish our results are similar to those in some previous literature, including, for example, the notion 
of "loop calculus" in pS QH EE [21] , and the "opening/closing the box" approach in [221 E3 [5] . We shall 
give proper credit to these authors as we proceed in this paper. 
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The remainder of the paper is structured as follows. Section [5] formalizes the exterior-function semantics 
for normal factor graphs, discusses its linear algebraic interpretations, introduces the notion of holographic 
transformations, and establishes the generalized Holant theorem. Section [3] discusses applications of holo- 
graphic transformations, including deriving the general normal factor graph duality theorem as a corollary 
to generalized Holant theorem, and explaining the principle of holographic reduction. The paper is briefly 
concluded in Sectional 

2 Normal Factor Graphs and Holographic Transformations 

The term "normal factor graph" has been used in the literature with various meanings. In particular, normal 
factor graphs as defined in [1] can be easily confused with normal graphs, normal factor graphs for codes, 
or graphs in which variables are represented by variable vertices. Making a joint effort with Forney |17j . 
we advocate in this paper a more rigorous use of the term "normal factor graph," and introduce a new 
semantics, the "exterior-function" semantics, 2 that defines what a normal factor graph means. To us, this 
new semantics is quite appealing, since it allows clean development of various graph properties, and has an 
elegant linear algebraic perspective. 

2.1 Normal Factor Graphs: The Exterior- Function Semantics 

Formally, a normal factor graph (NFG) is a graph (V, E), with vertex set V and edge set E, where the edge 
set E consists of two kinds of edges, a set E mt of ordinary edges, each connecting two vertices, and a set 
E CKt of "dangling edges," each having one end attached to a vertex and the other end free. 3 Each edge e € E 
represents a variable x e taking values from some finite alphabet X e \ sometimes we may alternatively say 
that the edge e represents the alphabet X e when we do not wish to specify the variable name. Each vertex 
v represents a complex- valued function 5 f v on the cartesian product Xe[ v ) '■= X e , where E(v) is the 

e£E(t) 

set of all edges incident to v. If we denote the set {/„ : v € V} of functions by fv, then the NFG is specified 
by the tuple (V, E int , E cxt J v ) . 

When treated as a factor graph under the conventional semantics [7], the NFG Q — (V, E lnt , E cxt , fv) 
represents the product Y[ fv(%E(v)) of all functions in fv- This product, expressing a function on Xe '■= 

v£V 

| X e , will be called the interior function 6 realized by the NFG. Here we have used the standard "variable 

set" notation xe( v ) to denote the set of variables {x e : e <E E(v)}. 

Now we introduce a new semantics for NFGs: instead of letting an NFG represent its interior function, 
we let it represent the interior function summed over all variables represented by regular edges. We call 

2 The same semantics is also presented in the concurrent development of Forney 1171 , in which what we call "exterior functions" 
are called "partition functions." 

3 More formally, such dangling edges are hyperedges of degree 1, and thus strictly speaking a NFG is a hypergraph rather 
than a graph. 

4 It is possible to generalize the results of this paper to the cases where the alphabets are infinite or uncountable, although 
this involves some subtle technicalities. 

5 All functions in this paper are complex- valued. However, all results can be generalized to F- valued functions, where F is an 
arbitrary field. 

6 In the conventional factor graph literature, an interior function is referred to as a "global function." 
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Figure 1: A normal factor graph (NFG) Q. 

this function the exterior function realized by the NFG. More precisely, the exterior function realized by the 
NFG Q = (V, E int , E cxt , f v ) is the function 

Zg(x E ^t) := ^2 II fv(x E (v))- (1) 

x E i„t v<EV 

Thus the exterior function involves only the external variables, represented by the dangling edges. Letting 
the NFG Q express the function Zq as defined in ([T]) is what we call the exterior-function semantics, which 
we will use throughout this paper. 

In this setting, one may view an NFG as an expression, or realization, of a function (the exterior function) 
that is given in "sum-of-products" form, as in ([1]), where each variable is involved in either one function or 
two functions, and the summation is over all variables that are involved in two functions. Since a variable 
involved in two functions (represented by a regular edge) is "invisible" in the exterior function, we call it 
an internal variable. In contrast, a variable involved only in one function (represented by a dangling edge) 
remains visible in the exterior function, and is called an external variable. 

An example of an NFG is given in Figure [TJ which realizes the exterior function 

Zg(x 1 ,X 2 ) = ^2 fi{x 1 ,X 3 ,X4)f2{x 3 ,X 5 ,X 6 )f 3 {x2,X4,X 5 ,X7,Xs)f4(xe,X 7 ,X 9 )f 5 {x 8 ,X 9 ). 

X 3 ,...,Xg 

At first glance, NFGs and this semantics may appear to impose a restriction on which sum-of-products 
forms are representablc. We note, however, that any sum-of-products form can be straightforwardly con- 
verted to one that directly corresponds to an NFG. This requires only that we properly replicate variables, 
using a "normalization" procedure similar to that of Forney in [I] for converting a factor graph to a normal 
graph. (The Appendix gives a detailed account of this procedure.) For this reason, using NFGs to represent 
sum-of-products forms entails no loss of expressive power. 
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Figure 2: Vertex Grouping/Splitting Procedure. Left to right: Vertex Grouping; right to left: Vertex 
Splitting. 

For notational convenience, we may denote the sum-of-products form in ([T]) by 7 

{h{x E (l))j2{x E (2)), ■ ■ ■ , f\V\{xE{\V\))) , 

if V is identified with the set {1, 2, . . . , \ V\}. Due to the commutativity of both multiplication and summation 
and the distributive law relating the two operations, it is easy to see that any ordering of the arguments 
of (•,-,...,•) expresses the same function. Consequently, we may write the sum-of-products form more 
compactly as (f v (xE( v )) '■ v € V), or even as (/„ : v G V), if no ambiguity results. 

2.2 Exterior-Function-Preserving Procedures 

The exterior-function semantics of NFGs allows us to identify immediately several elementary graph manip- 
ulation procedures that preserve the exterior function. 

Vertex Grouping/Splitting Procedure In a Vertex Grouping Procedure, two vertices representing func- 
tions / and g are grouped together, and the group is replaced by a vertex representing the function (f,g). In 
a Vertex Splitting Procedure, a vertex representing a function that can be expressed by the sum-of-products 
form (/, g) is replaced by an NFG representing (f,g). Figure [2] shows this pair of procedures. 

Lemma 1 Applying a Vertex Grouping or Vertex Splitting Procedure to an NFG preserves the realized 
exterior function. 

Proof: This lemma holds because these procedures simply correspond to conversions between sum-of-products 
forms (f,g,h\, . . . ,h m ) and ({f,g),hi,h2,..-,h m }, which evidently express the same function. □ 
We note that this pair of procedures were first introduced by Loeliger in [5j |6], who refers to Vertex 
Grouping as "closing the box," and to Vertex Splitting as "opening the box." However, in [3 [6] these 
procedures are used to interpret an NFG (in the original semantics) as a flexible hierarchical model, and to 
explain message-passing algorithms, rather than in the context of the exterior-function semantics. 

7 In the case of two arguments, say functions / and g, the sum-of-products form (/, g) should not be confused with the 
Hcrmitian inner product of / and g, whose definition requires complex conjugation of one of the two arguments. 
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Note that the Vertex Grouping Procedure may be applied recursively to an arbitrary number of vertices, 
say fi, /2, . . . , f m , so that these functions are replaced by a single function realized by /2, • ■ • , fm)- The 
resulting NFG still realizes the same exterior function. Similarly, the reverse Vertex Splitting Procedure also 
preserves the exterior function. For these reasons, when we draw a dashed box (as in Figure [2 left) to group 
some vertices, we may freely interpret the NFG as the equivalent NFG in which the box is replaced by a 
single vertex representing the function realized by the box. 

Equality Insertion/Deletion Procedure For any finite alphabet X, let 6= denote the {0, l}-valucd 
function on X x X which evaluates to 1 if and only if the two arguments of the function are equal. That 
is, S— is an "equality indicator function." In an Equality Insertion Procedure, a 8— function is inserted into 
an edge; in an Equality Deletion Procedure, a 6 = is deleted and the two edges originally connected to the 
function are joined. Figure [3] shows this pair of procedures. 

X X , , X 



Figure 3: Equality Insertion/Deletion Procedure. Left to right: Equality Insertion; right to left: Equality 
Deletion. The edges may be regular or dangling; the vertex labelled with "=" represents the function 6=. 

Lemma 2 Applying an Equality Insertion or Deletion Procedure to an NFG preserves the realized exterior 
function. 

Proof: This result is a corollary of Lemma [TJ and simply follows from the fact that if the 5= function is 
inserted into an edge incident to a function /, then we may group / with 8— and replace the sum-of-products 
form (f,S = ) with the function it expresses, namely /. □ 

Dual Vertex Insertion/Deletion Procedure Let X and y be two finite alphabets and $ : X x y — >• C 

and <fr : X x y — s- C be two functions. Then we say that $ and $ are dual with respect to the alphabet y, 
and call y the coupling alphabet, if ($(x,y), $(x',y)) = 8—(x, x') for every x, x' G X. In the case when X 
and y have the same cardinality, we also call the vertices representing $ and $ transformers, a term which 
will be justified in Section 12.31 In a Dual Vertex Insertion Procedure, we insert into an edge representing 
alphabet X a dual pair of functions $ : X x y — > C and $ : X x y — > C, with y being the coupling alphabet, 
and let the edge connecting the two functions represent y. A Dual Vertex Deletion Procedure is the reverse 
of a Dual Vertex Insertion Procedure, in which we delete a pair of dual functions and the edge connecting 
them, and then join the ends of the two cut edges. 

Lemma 3 Applying a Dual Vertex Insertion or Deletion Procedure to an NFG preserves the realized exterior 
function. 

Proof: The Dual Vertex Insertion Procedure is equivalent to first inserting a 8= function in the edge and 
then splitting the 8— function into a pair of dual functions. The lemma then follows from Lemmas [T] and [2] 
□ 
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Figure 4: Dual Vertex Insertion/Deletion Procedure. Left to right: Dual Vertex Insertion; right to left: Dual 
Vertex Deletion. The edges may be regular or dangling; the oppositely oriented triangluar vertices represent 
a dual pair of functions. 

2.3 A Linear Algebraic Perspective 

Before we proceed to introduce holographic transformations, we pause to interpret NFGs from a linear 
algebraic perspective. 

We will denote the set of all complex- valued functions on a finite alphabet X by C x . It is well known 
that <C X is isomorphic to the vector space C'^': after imposing an order on X, one can arrange the values 
of any function / 6 C x as a vector in C'^ according to that order. Similarly, depending on the structure 
of X, the function / may also be viewed as a matrix, or as its higher-dimensional generalization, namely 
a tensor; if X is the cartesian product X\ x X2 of some alphabets X\ and A2, then / may be regarded as 
a matrix; if A" is a multifold cartesian product of alphabets, then / may be viewed as a multi-dimensional 
array, or as a tensor. On the other hand, conventional linear algebraic objects like vectors, matrices and 
tensors may be alternatively regarded as multivariate functions. In particular, a tensor with n indices may 
be identified with a multivariate function involving n variables. From this perspective, any sum-of-products 
form corresponding to an NFG may be viewed as a linear algebraic expression. 

In the simplest case, consider a sum-of-products form {f{xi),g{xj)) involving exactly two functions 
/ : Xj — > C and g : Xj — > C, where I and J are two finite index sets, possibly intersecting, and where for 
every i £ I U J, Xi is an arbitrary finite alphabet. It is straightforward to verify the following propositions: 

• If I = J =/= 0, then (/, g) is the (non-Hcrmitian) vector inner product, or dot product, / • g, where / 
and g are regarded as | Xj | -dimensional vectors and "•" is a dot product. 

• If I D J 0, then (f,g) is the matrix-vector product / • g, where / is regarded as a |A?nj| x \Xj\ 
matrix, g is regarded as a \Xj\ -dimensional vector, and "•" is a matrix- vector product. 

• If / \ J, J \ I and If! J are all non-empty, then (/, g) is the matrix-matrix product / ■ g, where / is 
regarded as a |<-t/\j| x |A/nj| matrix, g is regarded as |A/nj| X |<-tj\/| matrix, and "•" is a matrix-matrix 
product. 

• If I and J are disjoint and both non-empty, then (/, g) is the vector outer product, matrix Kronccker 
product, or tensor product, / • g, where / and g are regarded as two vectors, two matrices, or two 
tensors, respectively, and "•" is the corresponding product operation. 

In summary, this simple sum-of-products form, namely (/, g), unifies various notions of "product" in 
linear algebra. This unification illustrates the convenience of understanding linear algebraic objects such 
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as vectors, matrices and tensors as multivariate functions, since in this perspective one never needs to be 
concerned with whether a vector is a row or column vector, whether a matrix is transposed, and so forth. 

A general sum-of-products form which involves multiple functions and which can be represented by an 
NFG may be viewed as a linear algebraic expression involving various such linear algebraic objects and 
various such notions of product. The fact that the sum-of-products form (•,-,...,•) does not depend on how 
its arguments arc ordered contrasts with the standard order-dependent notations in linear algebra. 

In this perspective, Lemma [T] follows from the order-independent nature of sum-of-products forms, and 
Lemma [2] follows from the fact that 8= is essentially an identity matrix. 

In linear algebra, vectors, matrices and tensors may be viewed alternatively as linear maps, which are 
characterized by the spaces they act on and the product operation used in defining the maps. Similar 
perspectives can be made explicit in the NFG context. For example, a complex-valued bivariate function 
f(x,y) defined on X x y may be viewed as two maps: when participating in the sum-of-products form 
(f(x, y),g(y)) with a function g : y — > C, / can be viewed as a linear map from the vector space to the 
vector space C x ; when participating in the sum-of-products form (f(x, y), h(x)} with a function h : X — > C, 
/ can be viewed as a linear map from the vector space C x to the vector space 

This aspect allows us to interpret dual functions in two different ways. Suppose that $ : X x y — > C and 
<I> : X x y — > C are a pair of dual functions. On one hand, we may view $ as a map from C x to and $ 
as a map from back to C x . In this view, the composition map $ o $ is the identity map from C x to C x , 
and $ is essentially the inverse or pseudo-inverse of $. On the other hand, we may view $ as a map from 
C x to and $ also as a map from C x to C^. In this view, the dot product of two vectors / and g in C x 
is preserved after they are mapped, respectively, to vectors (/, $) and (g, $) in C y . This view is justified by 
(/, g) = (/,$, = ((/,$), (#, <?>}. 

Finally, we justify the term "transformer" that was introduced in Section [2.2[ Suppose that the functions 
$ and $ on X x y are dual with respect to alphabet y, and that X and y have the same cardinality. 
Then we may identify $ with its square-matrix representation in which the rows are indexed by X and the 
columns are indexed by y; similarly, we may identify $ with its square-matrix representation in which the 
rows are indexed by y and the columns are indexed by X. The fact that $ and $ are dual with respect to y 
implies that the matrix product $ • $ is the identity matrix. Thus the matrix $ is the unique inverse of the 
matrix and vice versa. Therefore, the functions $ and $ may be regarded as a pair of transformations 
(or transformation kernels) that are inverse to each other. 

2.4 Holographic Transformations and Generalized Holant Theorem 

Now we are ready to define holographic transformations. 

Suppose that / is a finite index set and that for each i G I, there are two finite alphabets Xi and J- 7 ; 
having the same cardinality. We will call a function $ : Xj x yj — > C a separable transformation if $ is 
a transformation from C Xl to C y ' (namely, there exists a unique function $ : Xj x yj — > C such that 
($(x, y), $(x', y)) = 5 = {x,x') for all x, x' <E X), and there exists a collection of functions {3>^ € C^xy* ; { £ 
1} such that $ = Y[ Noting that Y[ ®i ma Y be identified with the sum-of-products form ($j : i E I), 

iGJ iel 

we see that transforming any function in / G C Xl by $ is equivalent to evaluating the sum-of-products form 
(f, ($j : i G /)), which can be performed via separately transforming / by each of the $i's in an arbitrary 
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Figure 5: Holographic transformation: an NFG Q (left) and its holographic transformation Q H (right). Each 
triangular vertex is a transformer, possibly different. Oppositely oriented transformers on an edge are the 
inverses of each other. 

order; hence the term "separable." 

It is easy to verify that if $ := Yl 1S a separable transformation, then its inverse transformation 

iei 

<!> : Xj x y>i — > C is Yi where each <£>i : Xi x J^i — > C is the inverse transformation of $j . It follows that if 

iei 

a transformation is separable, then so is its inverse. 

Holographic Transformation Let Q = (V, E mt , E cyit , fy) be an NFG where for each edge e G E := 
E cxt U E mt , X e is the alphabet of the represented variable. For each e G E, let y e be another alphabet 
having the same cardinality as X e . For every vertex v and every edge e G E(v), we associate a transformation 
<IVe : x^j -> C such that if e is a regular edge connecting vertices u and v, then $ Ujf , and ^^.e are 
the inverse transformations of each other. Let $„ := <Ev e for every vertex v. Locally transform each 

eEE(v) 

function f v to the function F v € C^ B ("' via F w := (/„, $„), and collectively denote {F v : v € V} by Fy . We 
call the NFG Q H := (V, E int ,E cxt , Fy) the holographic transformation of Q with respect to the collection of 
local separable transformations {Q v : v E V}. 

A graphical example of holographic transformation is shown in Figure [5] We note that holographic 
transformations keep the topology of the NFG unchanged, and only transform each local function. 

Theorem 1 (Generalized Holant Theorem) In the setting above, the exterior function Zqh of the NFG 

Q H is related to the exterior function Zg of the original NFG Q by 

ZgH(y E e*t) = (Zg(x E **t), (<S> e (x e ,y e ) : e G £ oxt )) , 
where for each e G E eyit , we have written <E> e in place of Q v , e - 

Proof: The theorem simply follows from Lemma [31 Graphically, as shown in Figure the holographic 
transformation is equivalent to first inserting into each edge an inverse pair of transformers and into each 
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dangling edge a transformer, and then transforming each local function by its surrounding transformers. 
Since each inverse pair of transformers cancels out, the only difference between the exterior function of Q 
and that of Q H is due to the transformers that have been inserted in the dangling edges. This establishes 
the theorem. □ 

When E cxt is the empty set, the exterior functions of Q and Q H reduce to scalars. In this case, the 
generalized Holant theorem reduces to the Holant theorem of [TTj . 

We note also that in the literature of "loop calculus" [HI HH1 HOI HI] , which has been introduced for the 
study of belief propagation and of the partition functions of statistical mechanics models, a result equivalent 
to the Holant theorem has been proved, and a transformation equivalent to our holographic transformation 
(on NFGs without dangling edges) has been proposed under the name of "gauge transformation" (see, e.g., 

ED). 

Although our generalization of the Holant theorem appears straightforward, we believe that there is a 
conceptual leap in this generalization. In particular, the original Holant theorem reveals only that there are 
redundant and structurally identical NFGs that may be used to represent the same scalar quantity as a sum 
of products; it makes no attempt to transform or reparameterize an exterior function that assumes a more 
general form. In the general setting of holographic transformations, when the original NFG Q is viewed 
as a realization of some function g := Zg on a collection of alphabets {X e : e <G E}, the holographically 
transformed NFG Q H is viewed as a realization of a related function g H := Zqh on a different collection of 
alphabets {y e : e G E}. In particular, the function g H may be regarded as a transform-domain representation 
of g via an "external change of basis;" namely, a change of basis for the vector space c'^™*', that is 
characterized by the "external transformation" ($ e : e <G E CKt ), where this "external change of basis" involves, 
in its sum-of-products form, a "local change of basis" for each component vector space C> x °>, e € E. 



3 Applications of Holographic Transformations 
3.1 Duality in Normal Factor Graphs 

The first duality theorem for codes on graphs was the normal graph duality theorem that was introduced 
by Forney in pQ. In the setting of pQ, the graphs considered, rather than being NFGs, are "normal graphs," 
where edges incident on one or two vertices represent "symbol" variables and "state" variables, respectively, 
and where each vertex represents a local group-code constraint. The global behavior represented by the 
graph is the set of all symbol-state configurations that satisfy all local constraints, and the graph itself 
represents a group code that consists of all symbol configurations that participate in at least one symbol- 
state configuration in the global behavior. In PQ, Forney introduced a local "dualization" procedure for 
normal graphs, which converts each local code constraint to its dual code constraint and inserts a "sign 
inverter" into every edge connecting two vertices. The normal graph duality theorem then states that the 
dualized graph represents the dual code. 

The normal graph duality theorem of pQ may be formulated as an equivalent theorem, which we call the 
"code normal factor graph duality theorem," using the language of normal factor graphs. More specifically, 
we may use an NFG to represent a state realization of a code C, where each vertex represents the indicator 
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function of a local code constraint, and the exterior function is, up to scale, 8 the indicator function of the 
code C. The dualization procedure may be reformulated on the NFG as converting the indicator function of 
each local code to the indicator function of the dual of the local code and inserting an indicator function 5 + 
into each edge, where 5+ evaluates to 1 if and only if the two arguments of the function are additive inverses 
of each other. Then the code normal factor graph duality theorem states that the exterior function realized 
by the dual NFG is up to scale the indicator function of the dual code C , 

In the framework of factor graphs, Mao and Kschischang [53] introduced the notions of multivariate 
convolution and convolutional factor graphs, and proved a duality theorem between a multiplicative factor 
graph and its dual convolutional factor graph. The duality theorem of [U] (Theorem 11), which we call 
the "MK theorem," states that a dual pair of factor graphs represent a Fourier transform pair. Since the 
indicator function of a code and that of its dual code are a Fourier transform pair up to scale, the code 
normal graph duality theorem and hence the normal graph duality theorem follow from the MK theorem as 
corollaries. 

In a concurrent development [T7] , Forney has established a general normal factor graph duality theorem, 
where the vertices of an NFG can represent arbitrary functions and the dualization procedure is defined 
as converting each local function to its Fourier transform and inserting a <5+ function into each edge. The 
general normal factor graph duality theorem states that a dual pair of NFG's represent a Fourier transform 
pair up to scale. This theorem reduces to the code normal factor graph duality theorem (and hence to the 
normal graph duality theorem) if each graph vertex is the indicator function of a local code. 

In this subsection, we will show that the general normal factor graph duality theorem follows directly 
from the generalized Holant theorem. 

Let g = (V, E int , E cxt ,f v ) be an arbitrary NFG, where each variable alphabet X e , e € E = E CKt U E int , 
is a finite abelian group written additively. It is well-known that every finite abelian group (X ', +) has a 
character group X A , consisting of precisely the set of all homomorphisms, called characters, of X mapping 
X into the multiplicative group of the unit circle in the complex plane. The character group X A of X has 
the following properties [25]. 

• The group operation + in X A is defined by (xi+X2)(x) = X\{x)x2{x) for any two characters &i,X2 G X A 
and any x G X. 

• (A" A ) A is isomorphic to X. This result, known as Pontryagin duality [25], allows each element of X to 
be treated as a character of X A . 

• X A is isomorphic to X. 

• For each x ^ X and x G X A , x(x) = x(x). We will denote 9 both x(x) and x(x) by Kx(x,x) and, for 
later use, denote nx{x, — x)/\X\ by kx(x,x). Keeping in mind that kx and kx are both defined with 
respect to the alphabet X ', we may sometimes suppress such dependency in our notation. It is easy to 
see that kx and Kx are a dual pair of functions (with respect to either X or X A ). Since X and X A 

8 The scaling constant is the number of symbol-state configurations in the global behavior that correspond to each codeword; 
this number is the same for every codeword since the global behavior is an abelian group. 

9 It is customary in the literature to denote both x(x) and x(x) by the pairing (x, x). But we choose not to use this notation 
since it collides with our notation for "sum-of-products" forms. 
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have the same size, they in fact define a pair of transformations, namely, the Fourier transform and its 
inverse, as we state next. 

• For any function / <G C , its Fourier transform T[f] is a complex- valued function on X A defined by 
•F[f] := (f,K). It follows that for any function / G C xA , its inverse Fourier transform J r_1 [/] is a 
complex- valued function on X, J r_1 [/] = (/, «). We note that the inverse Fourier transform operator 
J 7-1 may also be applied to a function / G C x and result in a function on X A , where in the sum-of- 
products form the summation is over the ^-valued variable. 

• If X is the direct product X\ x Xi of finite abclian groups X\ and X2, then X A is the direct product 
Xy x X2 of the character groups X£ and X£ . In this case, for any {x\,x<i) G X\ x X2 and any 

k((x!,X2), (XI,X 2 )) = k(xi,Xx)k(x2,X 2 ). (2) 

Equation ([3]) shows that the Fourier transform is a separable transformation. 
Returning to the NFG Q , we next obtain its "dual" by applying Forney's dualization procedure. 

NFG Dualization Procedure Replace each <Y e -valued variable x e by an A^-valued the variable x e . Re- 
place each function f v ,v G V, by its Fourier transform JF[fv\ and insert into each internal edge a vertex 
representing the function $+(■). Again, the function <5+ is an indicator function which evaluates to 1 if and 
only if its two variables are the additive inverses of each other. We will denote the resulting NFG by Q , and 
refer to it as the dual NFG of Q. 



Theorem 2 (General Normal Factor Graph Duality Theorem) The exterior function Z q realized by 
the dual NFG Q and the exterior function Zg realized by the original NFG Q are related by 

Z Q = \X E ^\-T[Z g \. (3) 

Proof: This theorem can be viewed as a corollary of the generalized Holant theorem, and can be simply 
proved graphically (Figure [5]) : Construct another NFG Q' by inserting a 8— function into each regular edge 
g- by Lemmas Zg, = Zg. Obtain the NFG Q' H from Q' by inverse Fourier transforming every 8 = in Q 1 
and Fourier transforming every other function. This corresponds to inserting the dual functions nx e and 
into each regular edge e (with adjacent to the function S-) and inserting the function kx f into each 
dangling edge e. Since the inserted transformers in each regular edge are the inverses of each other, this 
verifies that Q' H is a holographic transformation of Q' . By the generalized Holant theorem, 

Z g ,h = {Zg,, (Ke : e G £ cxt )) = (Zg, (ft e : e G £ oxt )) = T[Zg\. 

Invoking a well-known result that for 5— defined on X x X, J r_1 [(5 = ] = ppyi5+, we see that Q' H and Q are in 
fact identical except that in Q' , each 5+ inserted in edge e is scaled by prr- The theorem is then proved 
by collecting all the scaling factors. □ 
We note that Theorem [5] is the most general NFG duality theorem. If each function in an NFG is an 
indicator function of a local code, then the exterior function realized by the NFG is up to scale the indicator 
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Figure 6: NFG's Q (top left), Q (top right), Q' (bottom left) and Q' H (bottom right). 
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function of a group code. Such an NFG, which may be called a "code normal factor graph" ("code NFG") 
may then be used to represent the group code. This makes a code NFG equivalent to a normal graph. 
Since the indicator function of a code and that of its dual are (up to scale) a Fourier transform pair, the 
general normal factor graph duality theorem then reduces to the code normal graph duality theorem (and 
the normal graph duality theorem), which state that a dual pair of code NFGs (rcsp. a dual pair of normal 
graphs) represent a pair of dual codes. 

It is worth noting that the general normal factor graph duality theorem also follows from the MK theorem, 
via the application of the "projection-slice theorem" of the Fourier transform, or the "sampling/ averaging 
duality theorem" of [HJ Theorem 8]. However, the proof using the generalized Holant theorem seems more 
transparent. 

3.2 Holographic Reduction 

A holographic reduction may be regarded as a particular kind of holographic transformation applied to an 
NFG without dangling edges, by which a counting problem may be reduced to an equivalent "PerfMatch 
problem" . Using this technique, Valiant constructed polynomial-time solvers for various families of "count- 
ing" problems previously unknown to be in P . We now summarize the main results of Valiant and 
explain how holographic reduction works. 

The PerfMatch Problem Suppose that H = (V, E, w) is a weighted graph with vertex set V, edge set E, 
and weighting function w which assigns to each edge e <= E a complex weight w(e). The quantity PerfMatch 
tt(H) of H is defined as 

MeQ(H) e£M 

where Q(H) is the collection of all perfect matchings 10 of H . It is known that the PerfMatch problem, 
namely, solving for n(H), can be performed in polynomial time using the FKT algorithm j26[ [27] if if is a 
planar graph. 

A principle underlying holographic reduction is a graph-theoretic property which expresses the PerfMatch 
of a weighted graph as a sum-of-products form, in which each involved function is defined based on a local 
component of the graph. Such a local component is referred to as a "matchgate," and each involved function 
is referred to as the "signature" of such a matchgate. More precisely, a matchgate is a weighted graph H 
(which will be used as a local component of a larger graph) with a subset W of its vertices specified as its 
"external vertices" (which will be used to connect to other matchgates to form a larger graph). Suppose 
that (Hi,Wi), (H 2 , W 2 ), . . . , (H m , W m ) are a collection of matchgates. We may build a larger graph H by 
connecting these matchgates via their external vertices where the only restriction is that each external vertex 
of a matchgate {Hi, Wj) connects to exactly one external vertex of a different matchgate (Hj, Wj). The edges 
that connect the matchgates will be assigned weight 1. Figure [7] (a) shows two kinds of matchgates, which 
are used to construct a larger weighted graph as shown in Figure [7] (c) in such a way. 

10 In graph theory, a perfect matching of a graph is a set of non-adjacent edges such that every vertex of the graph is the 
endpoint of an edge in the set. 
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Now let £ denote the set of edges in the larger graph H that connect (the external vertices of) the 
matchgates, and let £(i) denote the subset of the edges in £ incident to the external vertices of the matchgate 
(Hi, Wi). Associate with each e £ £ a {0, l}-valued variable x e . The signature fa of the matchgate (Hi, Wj) 
is a function of the variable set Xsu) defined as follows: Every configuration Xsu,) is made to correspond to 
a subgraph of H L induced by deleting a subset of its external vertices; more precisely, an external vertex 
is deleted if and only if it is the endpoint of an edge e £ £(i) whose x e = 1 in the configuration Xg^y, the 
PerfMatch of the subgraph induced this way is then defined to be the value fa(x £ ^). Then it is possible 
to express the PerfMatch of the larger graph H as a sum-of-products form involving the signatures of the 
matchgates as follows. 

m 

xe i=l 

It is easy to verify that the sum-of-products form in (g]) has an NFG representation, since each variable 
x e ,e £ £, is involved in precisely two functions (noting that every edge e £ £ connects two matchgates). In 
this case, the NFG has no dangling edges, and the realized exterior function reduces to a scalar, i.e., the 
PerfMatch of the larger graph H. 

Figure [7] shows an example of how to build an NFG that realizes the PerfMatch of a graph using the 
signatures of its matchgates. In the figure, (a) shows two matchgates, where the signature of each matchgate 
by itself can be viewed as an NFG vertex in (b). When we use the matchgates in (a) to build the larger 
graph H in (c), then equality (0} suggests that the PerfMatch of H is realized by the NFG in (d). That is, 
the NFG topology is identical to the topology by which the matchgates form the larger graph H. Visually, 
the relationship between the NFG and the graph H is apparent: The picture in Figure E[d) is the NFG if 
we ignore the details inside the boxes, and is the graph H if we ignore the boxes. 

Solving Counting Problems via Holographic Reduction Many counting problems are described in 
terms of a large collection of variables and a large collection of constraints each involving a subset of the 
variables. The objective of such problems is to compute the total number of global variable configurations 
satisfying all the constraints. In this context, the idea of holographic reduction is to transform the problem 
of interest to a PerfMatch problem. We now outline this approach. 

1. Express the problem as the computation of the exterior function realized by a planar NFG Q without 
dangling edges. When this is possible, each vertex of Q represents an indicator (i.e., {0, l}-valued) 
function. 

2. For each variable x e inQ, find a pair of inverse transformations $ e and $ e , and construct a holographic 
transformation Q H of Q such that each function vertex in Q H represents the signature of a matchgate. 

3. Create a weighted graph H by replacing each vertex of Q H with the corresponding matchgate drawing, 
and assign weight one to each edge of Q H . This process essentially turns the holographically transformed 
NFG as in Figure [7Jd) into a weighted graph as in Figure [7£c) . 

By the Holant theorem, the exterior function realized by Q is the same as that realized by Q H . Since 
Q and Q H do not have dangling edges, the realized exterior function is in fact a scalar; by (gj), this scalar 
is the PerfMatch of H . It is easy to verify that Q being a planar graph implies that H is a planar graph 
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(c) (d) 

Figure 7: (a) Two matchgates, where solid circles represent the external vertices; (b) the signatures of the 
matchgates represented as NFG function vertices (the boxes); (c) a larger graph H constructed from the 
matchgates; (d) the NFG realization of the PcrfMatch of H, where each box is a function vertex. 
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(provided that each matchgate is also a planar graph). Thus solving the PerfMatch problem for H via the 
FKT algorithm solves the the original counting problem in polynomial time. 

In [TT], finding the "right" transformations {$ e : e <E E} that transform each local function in the original 
NFG to the signature of some matchgate appeared to be an art. Later Cai and Lu [13] presented a systematic 
approach to determine whether such a transformation exists. Remarkably, Cai et al. Q3] have extended the 
approach of holographic reduction beyond transformations to the PerfMatch problem by introducing the 
concept of "Fibonacci gates." 

4 Concluding Remarks 

Sums of products are fundamental in physics, computer science, coding theory, information theory, and 
indeed all of science and applied mathematics. In this paper, we have introduced what we call the "exterior- 
function semantics" for normal factor graphs, which establishes a one-to-one correspondence between sum- 
of-products forms satisfying certain nonrestrictive "normal" constraints and their associated normal factor 
graphs. Within this framework, we have introduced a very general notion of holographic transformations 
of normal factor graphs, and have stated and proved a general and powerful theorem (which we call the 
generalized Holant theorem) that relates the exterior function of a normal factor graph to that of its holo- 
graphic transformation. As corollaries of this theorem, we obtain Valiant's original Holant theorem, as well 
as a very simple proof of a general normal factor graph duality theorem, of which Forney's original normal 
graph duality theorem is a special case. This connection between two seemingly distant fields seems to us 
remarkable. 

Although the use of internal variables in graphical models is by no means new, the exterior-function 
semantics introduced in this paper seems to us elegant, intuitive, and potentially of wide application. The 
linear algebraic perspectives that we have mentioned briefly in this paper may have much more general use. 
Indeed, as Pascal Vontobel has observed [55], the use of "trace diagrams" in mathematical physics (see, e.g., 
[291 130j ) appears to have much in common with our graphical techniques. We suspect that the areas of 
potential applications are vast. 
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Appendix: Converting Arbitrary Sum-of-Products Forms to NFGs 

In the framework of factor graphs [7], the product of any collection of multivariate functions may be repre- 
sented by a factor graph. By specifying a subset of the variables in the factor graph to be "internal" (namely, 
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x 5 

Figure 8: The "marked" factor graph representing the sum-of-products form 

to be summed over), it is then possible to represent any sum-of-products form using a factor graph with 
additional marks on some variable vertices. 

Figurc[5]is an example of a sum-of-products form represented by such a "marked" factor graph, where the 
variable vertices marked with "x" represent internal variables; the variable vertices without such marks are 
external variables, namely, those remaining in the argument of the represented function. Such a "marked" 
factor graph then represents the product of all local functions with all internal variables summed over; the 
function resulting from the summation then clearly involves only the external variables, analogous to the 
exterior function of an NFG. 

Since a factor graph can have an unrestricted topology and an arbitrary subset of its variable verices may 
be marked, it is possible to represent any sum-of-products form using a "marked" factor graph. 

Without loss of generality, we will assume that there are no degree- 1 internal variable vertices in a 
"marked" factor graph, since otherwise it is always possible to modify the local function connecting to the 
variable by summing over that variable. 

The following procedure, operating on a "marked" factor graph representations, "normalizes" any sum- 
of-products form. 

Variable Replication Procedure. Suppose that a variable z in a factor graph has degree d. Then we 
may create d replicas {z±, Z2, ■ ■ ■ , Zd} of variable z, isolate z from its edges, and attach each of the d replicas 
to one of these edges. Remove z if z is an internal variable in the original factor graph. Connect all the 
replicas of z and z itself, if it is kept, to a new function vertex representing £=(■). Finally, mark all replicas 
of z internal (i.e., with "x"). Figure [S] is a graphical illustration of this procedure. 

A procedure similar to the Vertex Replication Procedure above was first presented in pQ. It is easy to 
verify that when applying the Variable Replication Procedure to any variable vertex, the sum-of-products 
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(a) Replicating an external variable 




(b) Replicating an internal variable 



Figure 9: Variable Replication Procedure. 
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Figure 10: The sum-of-products form resulting from normalizing the "marked" factor graph of Figure [5] 
Left: the sum-of-products form represented as a "marked" factor graph; right: the sum-of-product form 
represented as an NFG. 

form corresponding to the resulting "marked" factor graph expresses the same function as the original sum- 
of-products form does. 

On a "marked" factor graph, we may apply this procedure to every variable that does not satisfy the 
"normal" degree restriction (namely that an internal variable vertex have degree two and an external variable 
vertex have degree one). It is straightforward to verify that in the resulting "marked" factor graph, the 
normal degree restriction is necessarily satisfied by all variables. We can then represent the resulting sum- 
of-products form using the NFG notation, representing internal variables as edges and external variables as 
dangling edges. 

Figure [10] shows the sum-of-products form resulting from normalizing the "marked" factor graph in Figure 

m 
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